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Executive Summary 


The underrepresentation of women and racial and ethnic minorities in computer science (CS) and other fields of science, 
technology, engineering, and math (STEM) is a serious impediment to technological innovation as well as an affront 
to fundamental notions of fairness and equity. These gaps emerge in the early grades and tend to persist, if not widen, 
throughout the secondary and postsecondary years. The unconscious biases (UB) of teachers, school administrators, and 
fellow students may contribute meaningfully to the persistence of these gaps. Fortunately, a nascent literature on targeted 
interventions that directly address UB suggests there may be compelling opportunities to promote broader engagement in 
CS and STEM education and employment. 

The fields of neuroscience, social psychology, economics, and sociology articulate the many possible origins of UB 
and the ways in which UB can harm stereotyped groups, particularly in educational settings. This interdisciplinary literature 
yields two troubling, important insights: 

» Humans consciously and unconsciously store experiences in our brains and those experiences (memories) later 
influence instantaneous, automatic decision-making, which is critical to cognitive functioning and cannot be 
turned off. 

» Exposure to UB can trigger self-fulfilling prophecies by changing stereotyped groups' behaviors to conform to 
stereotypes, even when the stereotype was initially untrue. 

These insights provide specific guidance for mitigating the negative consequences of UB via interventions that disrupt 
the channels through which UB influences individuals and that highlight the insidiousness of UB, respectively. In particular, 
the following design insights should be considered when addressing UB systematically: 

» Asking individuals to "suppress biases” is likely to be counterproductive, as this requires a great deal of mental effort 
and can cause UB to eventually rebound above pre-intervention levels. 

» Teachers and classroom climate moderate the impact of UB, suggesting that teacher-facing interventions that 
carefully leverage the relevant psychological mechanisms (e.g., awareness, motivation, individuation, and empathy) 
have substantial promise to reduce teachers' UB and improve student outcomes. 

In sum, UB is a nontrivial problem in education, especially in CS and STEM education, and it is not easily addressed via 
traditional educational policies and interventions. However, interventions that identify and alter the frequently unconscious 
psychological processes that harm individuals' outcomes are currently being developed and piloted. Teacher-facing 
interventions, which can be administered to both pre- and in-service teachers, are particularly promising. In part, this 
is because by addressing UB among teachers, we can help shape the entire classroom context in supportive ways. 
Furthermore, teacher-facing interventions are potentially cost-effective and scalable, because infrastructure for teacher 
training is already in place. 

Still, much remains to be learned about the scalability, external validity, and optimal design of such interventions. 

Given the scope and complexity of the problem, interdisciplinary and inter-sector partnerships between public schools, 
universities, researchers, and industry will likely play a pivotal role in meeting these objectives. 
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Introduction 


A critical design feature of human decision-making is the 
tendency to make attributions about people and events 
subconsciously. Quick decisions, reflexively interpreting 
new information through existing patterns of thought, are 
sometimes necessary for human survival (Kahneman, 

2011). However, because these reflexive attributions 
are shaped by the broader social context, they can also 
constitute unconscious biases (UBs) that are an affront 
to fundamental notions of fairness. 1 UB exists in many 
contexts such as schooling, employment, the criminal 
justice system, and health care, particularly when most 
experts, gatekeepers, and authority figures are members 
of a privileged group. The insidiousness of UB is that it can 
lead to self-fulfilling prophecies that create and perpetuate 
inequities between groups, even when there was no pre¬ 
existing difference in ability and the UB (stereotype) was, 
by definition, incorrect. UB crosses multiple intersections 
of identity, including race, gender, class, sexual orientation, 
religion, and region. 

The presence of UB among teachers is likely to be 
particularly consequential. Teachers are on the front lines 
of society's efforts to promote equality of opportunity. They 
spend a substantial amount of structured time with children 
over their developmental trajectories. When the UBs of 
well-intentioned teachers influence their judgment towards 
particular students (e.g., by race, ethnicity, gender), it can 
influence their instructional practices, the expectations they 
convey, and their recommendations for relevant outcomes 
like course placement, special education, and discipline. For 
example, recent research indicates that non-Black teachers 
have significantly lower expectations of Black students 
(Gershenson, Holt, & Papageorge, 2016). Even subtle 
aspects of classroom environments, such as the gender 
ratio of students in a class or posters associated with 
masculine CS stereotypes, can trigger anxiety that affects 


the performance and academic engagement of females 
(a phenomenon known as "stereotype threat"). All of these 
factors can shape students' own attitudes and expectations 
about school, and recursively influence their field of study, 
educational attainment, and choice of employment. 
Moreover, such biases might provide insights into the 
mechanisms through which teachers, the most important 
school-provided educational input, affect long-run labor 
market success. UB may be particularly salient with respect 
to supporting student success in STEM and CS, fields that 
are important for technological innovation and economic 
growth but where persistent underrepresentation among 
female and Black and Hispanic students has been a long¬ 
standing concern. 

Encouragingly, however, a number of theoretically 
informed unbiasing interventions are being developed and 
tested. These interventions, which can be either student- or 
teacher-facing, have shown promise, at least in the narrow 
contexts in which they have been rigorously evaluated. The 
scalability of these interventions, and whether student- or 
teacher-facing interventions show more promise, are open 
questions we address later in this report. Specifically, the 
report sections proceed as follows: 

» Section 2, Understanding the Gaps in Education 
Outcomes documents current and past socio¬ 
demographic gaps in academic achievement, 
employment, and earnings in STEM and CS fields 

» Sections 3 and 4, Theoretical Explanations of 
UB/Implicit Bias and Evidence of UB describe 
theoretical and empirical evidence, respectively, on 
the existence and effects of UB in STEM and CS 
contexts 

» Section 5, Interventions to Address UB reviews the 
available credible field evidence on the efficacy of 
various theoretically-informed unbiasing interventions 

» Section 6, Conclusions and Future Directions 

concludes the report with a discussion of the most 
promising directions for future research, policy, and 
practice in this field 


1 Unconscious bias is also known as implicit social cognition (Greenwald & Banaji, 
1995). 
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Understanding the Gaps in Education 
Outcomes 


A diverse set of goals motivates both public and private 
investments in education. These include the desire 
to promote civic engagement and character as well 
as economic productivity. Another critical goal of 
education involves creating equality of opportunity for 
students regardless of socioeconomic and demographic 
backgrounds. However, there is also broad concern that 
schools may sometimes exacerbate, rather than remediate, 
inequality. Characterizing the gaps in different educational 
outcomes provides a useful framework for thinking 
about the policies and practices that might improve the 
educational outcomes of disadvantaged groups. We focus 
on Black, Hispanic, and female achievement since these 
groups are underrepresented in STEM and CS fields. 

In the United States, Black students enter public school 
with achievement in math and reading behind that of their 
White peers. These gaps at school entry can be explained 
by a modest set of controls for socioeconomic status (SES) 
(Fryer & Levitt, 2006). However, the gap widens as students 
progress through school, in ways that cannot be explained 
by socioeconomic status (SES) characteristics or measures 
of school quality. On entering school, Hispanic students 
also underperform relative to their White peers. This gap 
narrows somewhat in the first two years of school but then 
stabilizes (Reardon & Galindo, 2009). 

The recently released 2015 results from the National 
Assessment of Educational Progress (NAEP, "the Nation's 
Report Card") illustrate the considerable size of these gaps 
by subject and age. Among 4th graders, Black students 
perform below White students in mathematics by an 
amount equivalent to approximately 18 months of learning 
(0.80 standard deviations [SDs], Figure I). 2 By grade 8, 
this math gap has increased to 32 months of learning 
(nearly 0.87 SDs). Hispanic students underperform relative 
to White students by a slightly more modest amount. In 
reading (Figure 2), 4th grade Black students underperform 


relative to their White peers by 23 months of learning (i.e., 
0.70 SDs) and this amount increases to 34 months of 
learning by grade 8 (i.e., 0.74 SDs). For Hispanic students, 
these gaps are nearly as large. The reading achievement 
of Hispanic students is 22 months of learning (i.e., 0.65 
SDs) behind their White peers in grade 4 and this amount 
increases slightly to 28 months of learning by grade 
8. Interestingly, the math performance of girls is not 
meaningfully different from that of boys in the 2015 NAEP. 3 
However, boys significantly underperform relative to girls in 
reading. Specifically, at grade 4, the average performance 
of boys is substantially below that of girls (i.e., 0.19 SDs or 
approximately 6 months of learning) and, by grade 8, this 
gender gap has increased to roughly 1 full year of learning 
(i.e., 0.26 SDs). 

Figure 1. 

ACHIEVEMENT GAPS IN MATHEMATICS (STANDARD 
DEVIATION) BY RACE/ETHNICITY AND GRADE, 2015 NAEP 

MONTHS OF LEARNING 

35 



Grade 4 Grade 8 Grade 4 Grade 8 

White-Black White-Black White-Hispanic White-Hispanic 

Source: U.S. Department of Education 
Figure 2. 

ACHIEVEMENT GAPS IN READING (STANDARD DEVIATION) 
BY RACE/ETHNICITY AND GRADE, 2015 NAEP 

MONTHS OF LEARNING 

35 _ 



Grade 4 Grade 8 Grade 4 Grade 8 

White-Black White-Black White-Hispanic White-Hispanic 

Source: U.S. Department of Education 


2 In order to convert these achievement gaps into months of learning, we rely on 
the grade and subject-specific benchmarks provided by Hill, Bloom, Black, and Lipsey 
(2008). 


3 The lack of a qualitatively meaningful gender gap in mathematics is not due to 
a focus on means. At higher percentiles of math performance, young girls in these 
grades perform similarly to their male peers. 
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One explanation for the existence (and persistence) 
of large achievement gaps by race and ethnicity involves 
the effects associated with poverty (e.g., Lareau, 2011) 
and poverty's intersection with school quality. However, 
practices, policies, and social dynamics (e.g., unconscious 
biases among teachers) that vary within schools may 
also influence achievement gaps. One useful way to 
identify the broad sources of achievement gaps is to ask 
how much of these gaps can be explained by differences 
across schools (e.g., neighborhood or school quality) 
versus factors that vary within schools. For example, about 
half of the Black-White achievement gap is attributable 
to within-school sources (Bohrnstedt, Kitmitto, Ogut, 
Sherman, & Chan, 2015). This implies that within-school 
differences are at least as important as between-school 
differences associated with neighborhoods and school 
finances. Qualitative studies similarly suggest that differing 
experiences by race within schools (e.g., expectations of 
students, academic tracking, and school discipline) are 
educationally relevant (Lewis & Diamond, 2015). 

For example, about half of the Black-White achievement 
gap is attributable to within-school sources 
(Bohrnstedt, Kitmitto, Ogut, Sherman, & Chan, 2015). 

There are also racial gaps in behavioral outcomes. 

For example, the suspension rate for Black high school 
students is about three times larger than for White 
students. Boys are also twice as likely to be suspended 
as girls.Unsurprisingly, these gaps map onto gaps in 
educational attainment. For example, Figure 3 illustrates 
that graduation rates for Black students in public schools 
are 17 percentage points lower than that for White students 
(68% vs. 85%). A similar, but smaller, Hispanic-White gap 
in graduation rates is also apparent. Interestingly, the 
graduation rate for female students is seven percentage 
points higher than their male peers (85% vs. 78%). 


Figure 3. 

AVERAGE FRESHMAN GRADUATION RATE BY RACE, 
ETHNICITY, AND GENDER 

AVERAGE GRADUATION RATE (%) 
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Figure 4. 

COLLEGE MATRICULATION RATES OF RECENT 
HIGH SCHOOL COMPLETERS BY GENDER, 1975-2013 

COLLEGE MATRICULATION RATE (%) 



There are similar patterns in college matriculation: the 
college attendance rate for girls who have recently 
graduated from high school (Figure 4) is about five 
percentage points (8%) higher than that of boys. Also, about 
two-thirds of White high school graduates enroll in two- or 
four-year colleges soon after graduation, but this figure 
is 10 percent lower among Black and Hispanic graduates 
(Figure 5). 
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Figure 5. 

COLLEGE MATRICULATION RATES OF RECENT 
HIGH SCHOOL COMPLETERS BY RACE/ETHNICITY, 
1975-2013 


Figure 6. 

THE SHARE OF BACHELOR S DEGREES IN SELECTED STEM 
FIELDS EARNED BY FEMALES AND UNDERREPRESENTED 
MINORITIES, 2012 
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Female students perform as well as boys in K-12 math 
assessments, such as the NAER and have better outcomes 
in reading and in educational attainment. Furthermore, a 
recently released NAEP study that focused specifically on 
technology and engineering literacy among 8th graders 
found that girls, on average, performed somewhat 
higher than boys. 4 Nonetheless, females are dramatically 
underrepresented in important STEM fields. More 
specifically, while women received roughly 57 percent of the 
bachelor degrees awarded in 2012, they accounted for less 
than half of the bachelor degrees awarded in mathematics 
and statistics. More dramatically, they accounted for less 
than 20 percent of bachelor degrees awarded in computer 
science and engineering. There are similar but less stark 
gaps for underrepresented minorities, who received a 
combined 21 percent of the bachelor degrees in 2012, but 
received only about 12 percent of the degrees awarded in 
engineering, mathematics, and statistics and 19 percent of 
those in computer science (Figure 6). 


Source: National Science Foundation 

Overall, these measures suggest an interesting pattern 
in the gaps in educational outcomes. For minority students, 
the gaps in student outcomes start in early grades and 
persist throughout their educational trajectories. For girls, 
measured outcomes indicate that they are equivalent to 
(or outperforming) boys on a variety of metrics, including 
mathematics. However, in postsecondary schooling, girls 
stratify dramatically leading to underrepresentation in 
particular STEM fields. While this may seem to suggest that 
the sources of the "leaky pipeline" for women are situated 
in postsecondary settings, some of the relevant factors 
(e.g., expectations, confidence, mindset, secondary course 
taking, etc.) are formed in earlier K-12 settings. 


4 http://www.nationsreportcard.gov/tel_2014/ 
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Theoretical explanations of 
UB/Implicit Bias 


The fields of social psychology, neuroscience, economics, 
and sociology articulate the many ways in which 
unconscious bias can manifest, and harm negatively 
stereotyped (i.e., "outgroup”) populations. 5 Understanding 
the sources of UB and the channels through which UB 
affects outcomes such as academic performance, wages, 
and course of study can inform the creation of interventions 
that minimize UB's role in perpetuating inequities in CS and 
STEM fields. 

Stereotypes, or attitudes towards specific groups, do 
not necessarily reflect conscious thought (Devine, 1989). 
Indeed, individuals can be cognizant of stereotypes without 
believing them or consciously acting on them. However, 
unbeknownst to the individual, unconscious patterns of 
thought (e.g., reflexive assumptions about the academic 
motivation and capabilities of minorities and women) can 
influence current decisions. 6 These reflexive assumptions 
are the basis of UB. 

Due to their hierarchical structure, schools and 
universities may be particularly susceptible to the presence 
of UB. Stereotype threat (Steele, 1997) describes one 
process through which UB might shape the demographic 
composition of CS and STEM classrooms. Consider an 
introductory CS course taught by a male professor who 
has no overt prejudices and truly believes that all students, 
regardless of socio-demographic background, are capable 
of mastering the material. Without realizing it, however, 
the professor may behave in ways that trigger feelings 
of self-doubt among the female students. For example, 
he may signal lower expectations for female students by 
over-praising females for giving correct answers in class. 
The resultant stress may cause female students to lose 
interest in the course, put forth less effort, and ultimately 
choose not to major in CS. By causing female students to 

5 Our discussion of key concepts and insights is arranged by discipline. However, 
disciplinary boundaries are sometimes blurry. This is sometimes due to the rise in 
interdisciplinary research. In other cases, ideas have been independently discovered 
and pursued by researchers in different disciplinary silos. 

6 This is known as implicit social cognition (Greenwald & Banaji, 1995). 
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comport to negative stereotypes regarding females' ability 
and interest in CS, UB and its consequences can persist, 
because there are fewer female professors and students 
in CS classrooms. This scenario is particularly troubling 
because it shows how UB can create a self-fulfilling 
prophecy in which stereotyped groups underperform 
in CS and STEM fields, even when the initial UB-related 
stereotypes were incorrect. 7 

Meanwhile, advances in neuroscience have enhanced 
our understanding of many psychological phenomena, 
including UB. For example, we now know one region of the 
brain (the amygdala) regulates automatic "fast” responses, 
which are associated with UB, while another (the frontal 
lobe) regulates conscious (controlled) responses. Thus 
there are distinct neuro-processes underlying the thinking 
"fast" and "slow" dichotomy described in Kahneman (2011). 
Another important insight is that the automatic responses 
associated with UB are learned fear responses that are 
predicated on lived experiences (Amodio, 2014). In other 
words, humans are not born with an innate UB towards 
a particular group, but rather are exposed to a series of 
environments and experiences, which are both consciously 
and unconsciously stored in our brains, and later influence 
our instantaneous, automatic decisions. However, this is not 
to say that humans cannot reduce overall levels of UB, as 
we have some control over the experiences, environments, 
and individuals that we, and our children, encounter. 

Economics and sociology provide additional 
insights into how societal and organizational factors can 
promulgate UB and its consequences. Microeconomics 
is the study of how individuals make decisions, including 
such fundamental questions as whom to hire, where to 
live, what to study, and how much schooling to obtain. 
Economic theory provides insights into potential roles of 
UB in the startling, persistent sociodemographic disparities 
in STEM and CS educational and employment outcomes. 
First, statistical discrimination (Phelps, 1972) suggests 
that decision makers rely on the average ability of the socio¬ 
demographic group to which an individual belongs when the 
individual's own ability is unobservable (i.e., when employers 
have imperfect information about worker productivity). If it 
is difficult to ascertain a worker's ability prior to hiring them 

7 There are at least two ways in which UB might harm outgroup students' 
performance. First, behavior modification occurs when stereotyped groups modify 
their own behaviors to conform to negative biases. Second, teachers may modify how 
they teach, evaluate, and advise outgroup students, again leading to poor educational 
outcomes for such students. 
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and stereotyped minorities have objectively lower CS and 
STEM skills, perhaps due to the phenomenon of stereotype 
threat discussed above, rational employers in those 
sectors would respond by hiring few, if any, women and 
minorities. This would perpetuate the underrepresentation 
of stereotyped minorities in CS and STEM that triggered the 
phenomenon of stereotype threat in the first place. Similarly, 
if talented minority engineering students see themselves 
being passed over for internships or treated differently in 
classes, they may respond by making smaller investments 
in CS and STEM education. A feedback loop is apparent in 
both cases that can create a self-fulfilling prophecy in which 
initially incorrect stereotypes (due to UB) affect stereotyped 
groups' behaviors in ways that then make the stereotype 
become true (Loury, 2009). Second, identity economics 
(Akerlof & Kranton, 2005) articulates scenarios in which 
individuals simultaneously choose social identities (e.g., 
jock, nerd, burnout) and corresponding levels of academic 
effort that may reinforce the stereotyped identity. The UB of 
teachers, principals, and peers might affect school climate 
and students' expectations, effort, and attitudes, in ways 
that also reinforce such dynamics. 

The importance of school climate also appears in 
sociology's focus on organizational factors (Pager & 
Shepherd, 2008). Organizational factors likely shape the 
way that teachers' or peers' UB affects students (Reskin, 
2000; Petersen & Saporta, 2004). For example, social 
networks within schools or organizations might exacerbate 
the harm caused by UB if UB contributes to the formation 
of homogeneous "ingroup” (i.e., a group of people with 
a shared interest or identity) social networks, and such 
networks influence organizations' hiring and promotion 
decisions. 8 Again, this can lead to a perpetual lack of 
outgroup (minority) role models in leadership positions, and 
even trigger stereotype threat. 

Structural factors such as broader societal norms 
and policies also likely shape the influence of UB on 
individuals' choices, behaviors, and outcomes (Pager 
& Shepherd, 2008). Extant wage inequality, gaps in 
educational attainment, and occupational sorting by gender 
and race, which themselves are due to a combination of 
past explicit and implicit discrimination, mean that even a 
single instance of UB today, say when a deserving minority 
is denied a loan, can start a chain reaction that affects 
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housing, credit, children's educational options, wealth, 
and the intergenerational transmission of SES (DiPrete & 
Eirich, 2006). A similar chain reaction might occur when 
a student is counseled out of the advanced math track. 9 
In other words, transitory exposure to UB can expose 
outgroup populations to cumulative disadvantage, which in 
turn triggers their conformity to stereotypical or negatively 
biased expectations (Loury, 2003). 

In other words, transitory exposure to UB can expose 
outgroup populations to cumulative disadvantage, which in 
turn triggers their conformity to stereotypical or negatively 
biased expectations (Loury, 2003). 

Again, the common theme of these interdisciplinary 
theoretical insights is that UB can occur in numerous 
contexts and from various sources. And when it does, UB 
can create feedback loops and self-fulfilling prophecies 
that perpetuate inequities, and even create inequities where 
none existed before and the UB or stereotype was incorrect. 


9 Counseling-out occurs in public schools. For example, S. Nicholson-Crotty, 
Grissom, J.Nicholson-Crotty, and Redding (2016) show that Black students are 
significantly more likely to be assigned to gifted-and-talented tracks by Black teachers 
than by White teachers. 


8 Loury (2009) refers to this as "discrimination in contact." 
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Evidence of UB 


Cognitive Priming experiments provide indirect evidence of 
UB towards oneself, by showing that seemingly innocuous 
stimuli associated with stereotypes affects outgroup 
individuals' performance. For example, Steele and Aronson 
(1995) showed that Black college students perform worse 
on standardized exams that are framed as "diagnostic of 
intellectual ability" rather than a "laboratory problem-solving 
task." Johns, Schmader, and Martens (2005) found similar 
effects on female performance on math exams. Similar 
effects have been replicated across outgroup populations, 
educational contexts, and domains (Steele, Spencer, & 
Aronson, 2002). The idea here is that framing the exercise 
as a formal "test of ability" prompts students to recall 
stereotypes about females' and racial minorities' scholastic 
aptitude, which in turn creates stress that influences test 
performance. Situational contexts, such as classroom 
gender ratios, can produce similar effects by reminding 
female test takers that females are underrepresented in 
STEM (Inzlicht & Ben-Zeev, 2000; Bell, Spencer, Iserman, 

& Logel, 2003). This phenomenon is observed in other 
contexts as well. For example, the gender composition of 
the audience in a scientific conference affects females' 
willingness to participate in the conference (Murphy, Steele, 
& Gross, 2007). These findings show how seemingly 
innocuous stimuli can trigger subconscious psychological 
responses that affect performance in the CS and STEM 
domains. 

Implicit association tests (IAT), first proposed by 
Greenwald, McGhee, and Schwartz (1998), provide a 
more direct measure of UB. Importantly, lATs can detect 
attitudes that participants would consciously try to hide, 
or might not even be aware of. Numerous IAT studies find 
evidence of UB across contexts and domains (Nosek et 
al., 2007; Wilson, Lindsey, & Schooler, 2000). In education, 
lATs find that both genders unconsciously associate 
men with hard sciences and women with the liberal arts. 
Women with higher degrees of UB consciously report 
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being less interested in science and math, perform worse 
on standardized math exams, and are less likely to enroll 
in math and science majors (Nosek, Banaji, & Greenwald, 
2002). At the macro level, countries with high levels of 
gender-based UB towards science and math have larger 
country-level gender gaps in 8th grade math and science 
performance (Nosek et al., 2009). 

A limitation of the priming experiments and lATs 
discussed above is that they are typically conducted in 
sterile laboratory settings, and might not adequately reflect 
human behavior in the real world. Correspondence (resume 
audit) studies are one way around this: researchers submit 
fictitious applications to real job openings, providing a 
straightforward way to measure bias in high stakes, real- 
world scenarios (i.e., in the field). For example, Bertrand and 
Mullainathan (2004) sent fictitious resumes to job adverts 
in Boston and Chicago. The resumes were identical except 
for the "Whiteness" of the names randomly assigned to 
each resume. White names like Greg were about 50% more 
likely to be called back for interviews than Black names 
like Jamal. This result has been documented in numerous 
contexts and for an array of racial and ethnic minorities 
(Bertrand & Duflo, 2016). Correspondence studies that 
investigated gender discrimination in CS jobs found mixed 
results. Bertrand and Duflo call for further research that 
can sort out the mechanisms underlying these results. For 
example, Bartos, Bauer, Chytilova, and Matejka (2014) found 
that German and Czech employers called back ingroup 
applicants at higher rates and spent more time reviewing 
ingroup application materials. 

Correspondence studies have also been conducted in 
academic settings. Milkman, Akinola, and Chugh (2012) 
sent emails from fictitious prospective doctoral students to 
professors asking to meet. White male students received 
more, and faster, responses than female and non-White 
students in CS and STEM programs (Milkman, Akinoloa, 

& Chugh, 2015). This highlights the role that bias, whether 
explicit or implicit, can play even at informal or intermediate 
steps of the educational process. Similarly, Moss-Racusin, 
Dovidio, Brescoll, Graham, and Handelsman (2012) 
conducted a correspondence-type study in the lab, in which 
science faculty at research universities reviewed fictitious 
applications for a hypothetical lab assistant position. The 
scientists, regardless of gender, systematically rated male 
applicants as more competent than female applicants. 

It is striking that both male and female faculty were 




Unconscious Bias in the Classroom: Evidence and Opportunities 


equally biased against female candidates. It is difficult to 
disentangle UB from explicit or intentional bias in many 
of these studies, though given extant laboratory evidence 
on priming and lATs, it is likely that UB plays a nontrivial 
role in the discrimination documented in field experiments 
(Bertrand & Duflo, 2016). 

Grading biases provide additional evidence of 
discrimination in the classroom, although not always in 
the expected direction. In Israel, high school teachers 
assigned higher grades to females when the exams listed 
their names than on "blind" exams that hid students' 
names (Lavy, 2008). A U.S. study suggests that this 
unexpected gap is almost entirely explained by gender 
gaps in non-cognitive skills; that is, conditional on teachers' 
assessments of students' academic engagement, the male- 
female grading differential disappears (Cornwell, Mustard, 

& Van Parys, 2013). Hanna and Linden (2012) conducted 
a field experiment in India that randomly assigned names 
associated with different castes on exam cover sheets and 
found significant grading bias against children from lower 
castes. These results are troubling, as exposure to a biased 
teacher can harm several long-run educational and labor- 
market outcomes (Lavy & Sand, 2015). 

Grading biases are related to the consequences of the 
underrepresentation of females and racial/ethnic minorities 
in certain subjects and levels of the educational system. A 
seminal study by Dee (2004) found that having a same-race 
teacher significantly improved students' math and reading 
achievement. This result has been replicated in other 
educational contexts, as have effects of student-teacher 
gender match. Carrell, Page, and West (2010) showed 
that when female college students have a female math or 
science professor, they perform better in those classes and 
are more likely to engage with STEM subjects in the future. 

It is troubling, then, that De Paola and Scoppa (2015) found 
that female faculty are significantly less likely to be awarded 
tenure by all-male committees. These examples show how 
UB can create a "pipeline problem" that persists into the CS 
and STEM labor markets. 

There are similar effects of mismatch on student 
and teacher behaviors. In laboratory experiments, Gilliam, 
Maupin, Reyes, Accavitti, and Shic (2016) found that 
teachers had systematically graver perceptions of the 
severity of preschoolers' misbehavior when observing 
students of a different race other than their own race 
and that when primed to expect misbehavior, preschool 
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teachers of all races devoted more time to observing 
black students, especially black boys. Dee (2005) found 
that, when White and Black teachers simultaneously 
evaluated the same Black student, White teachers were 
significantly more likely to perceive the Black student 
as disruptive, inattentive, and less likely to complete 
homework. Gershenson et al. (2016) similarly showed that 
White 10th grade math teachers have lower educational 
expectations for Black students than do Black teachers. 
These biased expectations arguably have causal impacts 
on college completion rates that contribute to racial gaps in 
educational attainment (Papageorge, Gershenson, & Kang, 
2016). Demographic mismatch also affects intermediate 
outcomes associated with school engagement, such as 
student absences (Holt & Gershenson, 2015) and office- 
hours visits (Lusher, Campbell, & Carrell, 2015). Again, as 
discussed above in the context of priming, these behaviors 
create spillover effects that affect school climate and 
other outgroup students. Of course, like in the case of 
resume audits, it is possible that explicit biases contribute 
to the grading biases and impacts of student-teacher 
demographic mismatch discussed above, though UB likely 
plays an important role. 

These sorts of biases can have a variety of detrimental 
impacts and contribute to pipeline issues in CS and 
STEM. Most obviously, lower grades can affect students' 
admission into advanced academic tracks in secondary 
school and admission to postsecondary institutions. They 
can also indirectly affect student outcomes by changing 
how teachers advise and teach outgroup students. For 
example, a seminal experiment randomly manipulated 
teachers' expectations of students (i.e., identifying random 
students as likely to bloom in school), and students for 
whom expectations were raised subsequently experienced 
greater achievement gains (Rosenthal & Jacobsen, 1968). 
This result highlighted the potential for beliefs to create self- 
fulfilling prophecies, regardless of the accuracy of those 
beliefs, and has been replicated many times. These modest 
"Pygmalion Effects" have been replicated in numerous other 
contexts (Jussim & Harber, 2005). 10 

These instances of UB in K-12 and postsecondary 
schooling manifest in the underrepresentation of women 
and minorities in CS employment, though this is not 
entirely a pipeline issue; there is also UB in CS and STEM 

10 Pygmalion Effects are named after the character in Ovid's Metamorphoses who 
fell in love with a statue of a woman he created. 
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workplaces. For example, Reuben, Sapienza, and Zingales 
(2014) experimentally found that when employers were 
given no information about candidates other than physical 
appearance, men were twice as likely to be hired for a 
mathematical task as were women. Cheryan, Drury, and 
Vichayapai (2013) found that role models who fit CS 
stereotypes reduce women's interest in CS. For example, 
a White male instructor who fits the "computer nerd" 
stereotype of being interested in science-fiction movies 
and computer games can turn women off to the field. 

Even posters in the classroom that fit these stereotypes, 
and change classroom climate accordingly, can dissuade 
women from pursuing CS. 

For example, a White male instructor who fits the 
"computer nerd" stereotype of being interested in science- 
fiction movies and computer games can turn women off 
to the field. Even posters in the classroom that fit these 
stereotypes, and change classroom climate accordingly, 
can dissuade women from pursuing CS. 
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Interventions to Address UB 


School policies and practices 

A diverse array of federal, state, and local policies address 
socio-demographic gaps in educational outcomes. Few 
policies focus specifically on UB. Instead, they address the 
hypothesized consequences of UB. For example, a report by 
the National Research Council (2011) found that test-based 
school-accountability reforms had modest, positive effects 
on math achievement. Tracking is another prominent 
school practice, which sorts students into classrooms 
based on past performance, and likely exacerbates 
inequality (Hanushek & Woessman, 2006). A recent study 
by Card and Guiliano (2016) is consistent with the role of 
access to appropriately challenging curricula. They found 
that non-gifted Black and Hispanic students experienced 
achievement gains when granted access to classes for 
gifted students, perhaps due to UB-related factors such as 
teacher expectations. 

In a study of three Los Angeles high schools, Margolis, 
Estrella, Goode, Holme, Nao (2010) document how 
student underrepresentation in CS reflects the interplay 
of school policies (e.g., course offerings, academic rigor, 
student counseling) with teacher beliefs and behaviors. 
Classroom practices such as having positive role models 
administer exams might also minimize the harm of UB in 
engineering education (Eschenbach, Virnoche, Cashman, 
Lord, & Camacho, 2014). This can counter the pernicious 
effects of stereotype threat. Teachers can also design 
physical classroom environments so as to avoid prompting 
traditional stereotypes about engineering and CSthat 
may trigger social identity threat (Cheryan, Plaut, Davies, & 
Steele, 2009). 

Student-facing interventions 

Numerous studies have investigated how efforts to "buffer" 
students against the psychological effects of stereotype 
threat affect academic performance. The design of these 
interventions is based on theoretical insights gleaned 
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from lab experiments and aims to reduce the harm of UB 
rather than the existence of UB. We reviewed "buffering" 
interventions that (i) were conducted in field settings, (ii) 
used experimental methods, and (iii) focused on student 
outcomes. The 21 peer-reviewed studies fit in four mutually 
exclusive categories (Table I). 11 

Most (n = 12) were "values affirmation" (VA) exercises. 
These student-led interventions involved completing 
a brief worksheet in which students write about their 
personal values. A second set of studies (n = 5) focused on 
engagement with external attributions (EA) (e.g., luck, and 
other factors outside the student's control) for academic 
difficulties. A third set of studies (n = 5) involved influencing 
student “mindsets" (MS). These interventions encouraged 
students to adopt "growth" mindsets that emphasize the 
malleability of achievement over fixed mindsets that do 
not. The remaining study examined a "task reframing" (TR) 
intervention (Good et al., 2008) that involved describing a 
test in a manner that effectively disarmed stereotypes. 

These studies, mostly conducted in real-world middle- 
school and college settings, generally found large effects 
(e.g., 0.2 to 0.3 SDs) on academic achievement. Two 
studies of female STEM undergraduates found uniquely 
large effects (Miyake et al., 2010; Walton, Logel, Peach, 
Spencer, & Zanna, 2015). Because these interventions 
are brief and virtually costless, their cost-effectiveness is 
extraordinarily high. 

However, the impact of these interventions is theorized 
to rely critically on triggering "recursive" processes (Yeager 
& Walton, 2011). That is, these interventions are a type of 
"nudge" that relies on supportive school and classroom 
contexts to amplify a modest initial treatment into more 
dramatic and sustained change. Many of the studies in 
Table 1 rely on small numbers of teachers and schools. For 
example, the influential study by Cohen, Garcia, Apfel, and 
Master (2006) involved just three teachers in one middle 
school. There is some evidence that many studies were 
conducted in favorable school and classroom settings: 
effects were smaller in studies with larger numbers of 
students (e.g., Paunesku et al., 2015; Harackiewicz et al., 
2014) and non-existent in studies with the largest numbers 
of students, teachers, and schools (Dee, 2015, Lauer et 
al., 2013). These larger studies may have better external 

11 These studies could also be situated in a broader field-experimental literature 
based on motivation theory. In a recent meta-analytic review, Lazowski and Hulleman 
(2015) find that these interventions generally produce quite large educational gains 
(i.e., mean effect size of 0.49 SDs). 
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validity and likely included contexts not as conducive to 
such interventions. Dee (2015) provided direct evidence 
on this question in a values-affirmation study conducted 
in 128 classrooms across six middle schools. Overall, the 
researcher found no effect of the intervention. However, 
Dee did find sizable effects in classrooms with effective 
teachers (as measured by their value-added scores). 12 
These results do not impugn the theoretical basis of 
student-facing interventions but do raise serious doubts 
about their context-dependent scalability. 


12 Value-added scores are statistical estimates of an individual teacher's average 
contribution to student learning, as measured by standardized tests. See, for example, 
Chetty, Friedman, and Rockoff (2014). 
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Teacher-facing Interventions 

There are fewer rigorous evaluations of teacher-facing 
efforts to reduce UB. However, policies and programs 
that target teacher bias directly are attractive relative 
to both school and student-level approaches for three 
reasons. First, engagement at the teacher level reaches 
into classroom practice in a way that higher-level policies 
do not. Second, they engage the entire classroom context, 
creating the possibility of supportive "recursive" processes 
(e.g., by improving classroom climate or creating positive 
peer interactions) and, by implication, more promising 
scalability. Third, the established infrastructure associated 
with teacher training and professional development (PD) 


Table 1. 

STUDENT-FACING SOCIAL-IDENTITY INTERVENTIONS 


Authors 

Study Population (Sample Size 

Intervention 

Effects 

Walton and Cohen 
(2011) 

College students (92) 

EA 

0.30 GPA points for Black students 

Stephens et al. (2014) 

College students (168) 

EA 

0.24 GPA points 

Yeager et al. (2014) 

High school students (76) 

EA 

0.34 SD in grades for Black students 

Good et al. (2003) 

7th graders (138) 

EA/MS 

0.52 to 0.72 SD (reading); 1.1 to 1.5 SD (math) 

Walton et al. (2015) 

College students (228) 

EA/VA 

1.04 SD increase in women's engineering GPA 

Oysterman et al. (2016) 

8th graders (264) 

MS 

0.23 GPA points 

Blackwell et al. (2007) 

7th graders (91) 

MS 

0.53 GPA points in math 

Yeager et al. (2014) 

9th graders (78; 150 in 2 studies) 

MS 

0.34 GPA points in study 2, no effect in study 3 

Paunesku et al. (2015) 

High school students (1,594) 

MS 

0.14 GPA points for at-risk students 

Good, Aronson, and 

Harder (2008) 

College students (157) 

TR 

0.36 SD in math test for White females 

Cohen et al. (2006) 

7th graders (243) 

VA 

0.34 GPA points for Black students 

Cohen et al. (2009) 

7th graders (416) 

VA 

0.24 GPA points for Black students 

Woolf etal. (2009) 

Medical-school students (348) 

VA 

-0.18 SD in written assessment for White students, 
0.28 SD increase in test performance 

Miyake et al. (2010) 

College students (399) 

VA 

0.93 SD for females on physics exams 

Cook etal. (2012) 

7th graders (121) 

VA 

GPA gain for Black students 

Bowen et al. (2013) 

6th to 8th graders (274) 

VA 

0.57 SD gain in social studies grade 

Lauer et al. (2013) 

College students (679) 

VA 

No effect 

Sherman et al. (2013) 

6th to 8th graders (199) 

VA 

0.25 GPA points for Hispanic students 

Harackiewicz et al. 

(2014) 

College students (798) 

VA 

0.10 GPA points (biology) for first-generation 
students 

Dee (2015) 

7th and 8th graders (2,564) 

VA 

No effect 

Borman et al. (2016) 

7th graders (1,012) 

VA 

0.082 GPA points for minority students 


Notes: This table summarizes the findings from field-experimental interventions that focus on social identity mechanisms. 
The taxonomy of these interventions consists of values affirmation (VA), external attribution (EA), mindset (MS), and task 
reframing (TR). 
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provides opportunities to situate such interventions for both 
pre-service and in-service teachers. 

Traditional pre-service teacher training programs might 
directly or indirectly reduce UB and its harm. Recently, 
educational experts have argued that all education- 
program curricula should be augmented to include cultural 
competency training (NEA, 2008) and an explicit focus on 
race (Milner & Self, 2014). However, rigorous evidence on 
the precise pre-service curricula that reduce UB and its 
harms is lacking. 13 Non-traditional (alternative) teacher 
training programs such as Teach for America (TFA) also 
might address UB. Indeed, TFA's five-week summer institute 
includes an entire unit on the power of high expectations, 
and TFA teachers improve students' math and science 
achievement. 14 While the importance of TFA's focus on high 
expectations cannot be disentangled from that of other 
institutional features, TFA training does affect graduates' 
beliefs about the malleability of achievement gaps and UB 
(as measured by IAT scores; Dobbie & Fryer, 2015). 

A diverse literature in social psychology provides more 
explicit guidance on how to design interventions that 
may attenuate UB among teachers. For example, Cohen, 
Steele, and Ross (1999) showed that Black students 
responded particularly well to critical feedback when it was 
accompanied by a statement of high standards coupled 
with assurances about the students' capacity to meet those 
standards (i.e., "wise" feedback; Walton, 2014). Yeager 
et al. (2014) examined the effects of this approach in a 
field-experimental setting in which teacher feedback was 
experimentally manipulated. They found positive effects 
of wise feedback on measures of student engagement 
and performance, particularly for Black students who were 
mistrustful of school. 

A broader psychological literature on reducing UB (i.e., 
not necessarily in education) focuses on several design 
themes: 15 

» Nurturing the motivation to reduce UB by building 
an awareness of one's own biases without shaming 
or blaming (Devine & Monteith, 1993) 


13 Whether and how the location of teachers' pre-service teaching experience 
matters is another area of debate. Ronfeldt (2012) finds that teachers who have 
student-teacher assignments in easier-to-staff schools tend to have higher retention 
rates and higher value-added scores than teachers whose initial student-teaching 
placement is in a hard-to-staff school, though it is unclear whether this is a causal 
relationship. 

14 See Glazerman, Mayer, and Decker (2006) and Xu, Hannaway, and Taylor (2011). 

15 Burgess, Van Ryn, Dovidio, and Saha (2007) discuss this taxonomy in the context 
of reducing UB in medical settings. 


» Building awareness of the shared psychological 
basis for UB (Burgess et al., 2007) 

» Promoting evaluating individuals through 

individuation (unique attributes) rather than social 
categorization (group membership) (Blair, 2002) 

» Reducing the anxiety of outgroup interactions 
through increased contact between two or more 
social groups (Schellhaas & Dovidio, 2016) 

» Enhancing emotional-regulation skills that promote 
positive emotions when interacting with outgroups 
(e.g., visualizing the "Best Possible Self"; Sheldon & 
Lyubomirsky, 2006) 

» Increasing empathy and perspective-taking 

(Dovidio et al., 2004; Okonofua, Paunesku, & Walton, 
2016) 

» Building a sense of partnership that reduces 
outgroup status (Dovidio et al., 2004) 

Some of the research on these design features have 
been situated in education settings. For example, Okonofua 
et al. (2016) evaluated an intervention that encourages 
middle school teachers to consider and value student 
experiences. The study confirms that empathy is malleable 
and that teachers who use empathic discipline develop 
better relationships with outgroup students. Ultimately, 
student suspensions fell by 50%. 

Other education-specific studies leveraged an insight 
from Devine, Forscher, Austin, and Cox (2012): effective UB 
interventions may combine some or all of the mechanisms 
enumerated above. For example, Jackson, Hillard, and 
Schneider (2014) found that a multi-faceted diversity 
training session reduced UB towards women among male 
STEM faculty in postsecondary institutions. Similarly, 

Carnes et al. (2015) found that a 2.5 hour workshop that 
embeds these psychological mechanisms increased an 
awareness of personal biases among STEM faculty at the 
University of Wisconsin. 
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Conclusions and Future Directions 


Section 2 of this report documented troubling, persistent 
socio-demographic gaps in students' educational 
attainment and achievement in the CS and STEM fields 
that manifest in similar gaps in wages and occupational 
choice in the labor market. Section 3 introduced implicit 
or unconscious bias (UB) as a likely driver of such gaps 
and explained the theoretical underpinnings of the 
existence, and negative implications, of UB. Section 4 
then documented evidence of the harmful effects of UB 
in CS and STEM education and labor markets. Finally, 
section 5 reviewed the credible evidence on the efficacy 
of field-tested student- and teacher-facing educational 
interventions to reduce UB and its associated harms. 

Moving forward, the research agenda should 
privilege teacher-facing interventions over student-facing 
interventions for several reasons. This may seem odd as 
there is a thicker field-experimental literature on student¬ 
facing interventions and they show a great deal of promise. 
However, these field studies are typically situated in small, 
select settings and there is broad acknowledgment that the 
effectiveness of these student interventions relies critically 
on classroom context. Because many pilot studies of such 
programs were conducted in schools and classrooms 
with climates conducive to such interventions, the external 
validity of the results of these pilot studies is unclear. 
Interventions that engage teachers have the potential to 
improve classroom, and even school, climate in ways that 
increase the efficacy of many educational interventions and 
inputs. For example, interventions that reduce teachers' 
expressions of UB might also indirectly reduce ingroup 
students' expressions of UB. Additionally, teacher-facing 
interventions are more logistically feasible than student¬ 
facing interventions because there are fewer teachers, their 
turnover is lower, and they have established opportunities 
for PD training, both pre-service and in-service. 

Moreover, the separation between student- and teacher¬ 
facing interventions can be counterproductive. For example, 
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if buffering interventions, which are typically viewed as 
student-facing, are only effective when implemented in 
certain types of classrooms, then an intervention that 
trains teachers to implement such interventions or to 
organize their classrooms in a way that is conducive to 
such interventions, is both student- and teacher-facing 
and might yield large benefits. Future research on this type 
of hybrid intervention would be useful, as it represents a 
more holistic approach to addressing UB. For example, 
there may be multiplicative effects of interventions that 
simultaneously train teachers to empathize with students, 
build warm classroom climates, and implement affirmation 
exercises that are larger than the sum of effects when 
those interventions are offered independently. This is the 
type of innovative, potentially high-reward intervention that 
a partnership between schools, researchers, and industry 
could implement and evaluate. 

It is important that well-established and tested theories 
inform the design of teacher-PD programs. In particular, 
interventions that encourage teachers to see students as 
individuals instead of as categories and that build empathy 
and high expectations (e.g., visualizing best possible 
academic selves) are particularly promising (Carnes et al., 
2015; Jackson et al., 2014; Okonofua et at, 2016). However, 
the design of such teacher PD also needs to be mindful of 
the unintended negative consequences from an intervention 
that clumsily suppresses biases, as this approach can lead 
to unproductive rebound effects. 

While there are compelling opportunities in this space, 
the research base is, at present, too thin to guide policy and 
practice reliably. For example, there are emerging models 
of teacher training that hold the promise of reducing UB 
(e.g., simulation-based modules that provide pre-service 
teachers with practice interacting with minority students 
in challenging, yet common, classroom situations). 

However, such interventions have yet to be rigorously 
evaluated or implemented at scale. Technology and other 
CS-related industries are uniquely positioned to partner 
with schools and researchers to design, implement, and 
evaluate emerging interventions in this mold and ultimately 
increase the representation of female and racial and ethnic 
minorities in the STEM and CS sectors. 

Specifically, the near-term agenda should focus on 
an energetic design cycle in which theoretically-informed 
interventions are designed and rigorously evaluated in field 
settings. In the case of CS and STEM education, this means 
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at the primary, secondary, and postsecondary levels and in 
multiple school contexts (e.g., by demographic composition 
and diversity, students' socioeconomic background, subject, 
teachers' experience, and school and class size). One 
potentially important design element, which has yet to be 
carefully considered, is the medium of the intervention. For 
example, one cost-saving innovation might be to replace 
(or augment) simulations and classroom-experiences with 
role-playing video games or virtual-reality environments. 
Another novel, potentially important design element to 
experiment with is the frequency and duration of teacher¬ 
facing interventions. For example, the workshops evaluated 
by Carnes et al. (2015), Jackson et al. (2014), and Okonofua 
et al. (2016) need not be one-off events, but could be 
offered on an annual or monthly basis. Shorter, perhaps 
virtual, workshops and simulations could be offered even 
more frequently. 

In sum, the time is ripe for thoughtfully targeted 
and comprehensive action. A large and diverse body of 
evidence indicates that UB among teachers contributes 
meaningfully to education inequality. And, because this is 
a particular problem in CS and STEM fields, UB effectively 
deprives society of the talent and ingenuity that drives 
technological progress. Fortunately, emerging research 
indicates that there are likely to be effective ways to reduce 
the exclusionary effects of UB on CS and STEM education 
and employment. Flowever, this effort will require the 
cooperation and coordination of inter-disciplinary, inter-sector 
teams that thoughtfully design, rigorously evaluate, and 
implement on a wide scale interventions targeted to both 
pre- and in-service teachers. 
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About Google 

Google's core mission is to organize the world's information 
and make it universally accessible and useful. Google 
creates products to increase access to opportunity, break 
down barriers and empower people through technology. 

To help reach these goals, Google works to inspire young 
people around the world not just to use technology but 
to create it. There is a need for more students to pursue 
an education in computer science, particularly girls and 
minorities, who have historically been underrepresented in 
the field. More information on Google's computer science 
education efforts is available at g.co/csedu. 


About CEPA 

The Stanford Center for Education Policy Analysis 
(CEPA) is a research center created in 2009 to unite an 
interdisciplinary array of nationally prominent scholars 
from across the campus to provide the depth and scale of 
research needed to affect education practice and policy in 
meaningful ways. The CEPA community includes Stanford 
faculty, post-doctoral fellows, graduate and undergraduate 
students as well as visiting scholars and students from 
across the world. For more information about CEPA, visit 
cepa.stanford.edu 


About SPA 

American University School of Public Affairs is a top-ranked 
school offering programs to build and enhance careers 
in public service. The school offers a unique pairing of 
access to Washington, DC with world-renowned faculty 
and transformational research—driving progress in policy, 
politics, and public administration. SPA is ranked fifth in 
the world, third in the US, and first in the DC area for public 
affairs research impact. For more information about SPA, 
visit www.american.edu/spa. 
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